Image Captioning with Context-Aware Auxiliary Guidance

Image captioning is a challenging computer vision task, which aims to generate natural language description of an image. Most recent researches follow the encoder-decoder framework depends heavily on previous generated words for current prediction. Such methods can not effectively take advantage future predicted information learn complete semantics. In this paper, we propose Context-Aware Auxiliary Guidance (CAAG) mechanism that guide model perceive global contexts. Upon model, CAAG performs semantic attention selectively concentrates useful predictions reproduce generation. To validate adaptability method, apply three popular captioners and our proposal achieves competitive performance Microsoft COCO image benchmark, e.g. 132.2 CIDEr-D score Karpathy split 130.7 (c40) official online evaluation server.